Skip to content

feat(agent)!: align skill install with the Agent Skills standard#271

Open
powxenv wants to merge 8 commits into
TestSprite:mainfrom
powxenv:chore/agentskills-standard
Open

feat(agent)!: align skill install with the Agent Skills standard#271
powxenv wants to merge 8 commits into
TestSprite:mainfrom
powxenv:chore/agentskills-standard

Conversation

@powxenv

@powxenv powxenv commented Jul 22, 2026

Copy link
Copy Markdown

What does this PR do?

Aligns agent install with the Agent Skills open standard. Instead of hand-wiring a few agents with bespoke formats, the CLI now writes one canonical skill per project at .agents/skills/<skill>/SKILL.md (the standard's shared directory), and each agent reads it directly (universal agents) or via a symlink back to it. The bundled registry grows from 8 targets to the full standard set (72 agent ids), so any skills-compatible agent works out of the box.

This broadens adoption (works with 72 agents instead of 8), simplifies maintenance (one uniform code path instead of per-agent wrap()/mode/compact-body machinery), and follows a vendor-neutral standard already adopted by the agents users run.

Related issue

Closes #270

Type of change

  • Bug fix (non-breaking change that fixes an issue)
  • New feature (non-breaking change that adds functionality)
  • Breaking change (fix or feature that changes existing behavior)
  • Documentation only
  • Build / CI / chore

Checklist

  • PR targets the main branch.
  • Commits follow Conventional Commits
    (feat(...), fix(...), docs(...), …).
  • npm run lint and npm run format:check pass.
  • npm run typecheck passes.
  • npm test passes and coverage stays at or above the 80% gate.
  • New behavior is covered by unit tests (mock-based; no network or
    credentials required).
  • No secrets, API keys, internal endpoints, or personal data are included.
  • User-facing changes are reflected in README.md / DOCUMENTATION.md where
    relevant.

Notes for reviewers

Breaking changes

This restructures the install model, so existing behavior changes for current users:

  • Per-agent bespoke formats removed. Codex no longer receives a managed section in AGENTS.md; Windsurf no longer receives a size-capped compact body. Both now read the canonical .agents/skills/ (codex directly, windsurf via symlink), like every other agent. The wrap() / own-file / managed-section machinery is gone in favor of one standard SKILL.md.
  • Install output schema changed. agent install results carry a new mode (canonical | symlink) field, and path now reflects the canonical-or-symlink model rather than each agent's bespoke location.
  • Existing installs go stale. Skills written in the old format will surface as stale/modified under agent status until refreshed — re-running agent install (with --force where needed) brings them in line.

What's preserved: the claude / kiro / copilot aliases still resolve, and agent install with no --target still defaults to claude-code — so the commands existing scripts use keep working; only the on-disk layout and output change.

Implementation notes

  • Model: canonical .agents/skills/<skill>/SKILL.md is the single source of truth. Universal agents (Codex, Cursor, Cline, Gemini CLI, Copilot, …) read it directly; every other agent gets a relative symlink from its own skills folder back to it — no drift between agents.
  • Path safety: both the canonical write and the symlink landing walk each path component with lstat and refuse to traverse a planted symlink (exit 5), so a malicious symlink can't redirect a write outside --dir.
  • Tests: the install/status/conflict/force/dry-run/path-safety paths are covered exhaustively by unit tests using an in-memory filesystem (deterministic, cross-platform, no real-symlink fragility); coverage on agent.ts is ~92% lines, with the remainder being defensive error handling and real-fs wrappers exercised by the e2e suite.

Summary by CodeRabbit

  • New Features
    • Added supported-agent guidance with target aliases and universal vs symlinked install behavior (including Windows copy fallback).
    • Updated agent install to use a shared canonical skills source, with improved list/status output, dry-run planning, and clearer conflict + --force overwrite/backup behavior.
    • Default setup agent is now claude-code, and onboarding/verification skills include clearer metadata.
  • Documentation
    • Reworked CONTRIBUTING and all onboarding docs to match the new --target install model, supported-agents tables, and status semantics.
  • Bug Fixes
    • Improved the “skill installed” nudge detection to rely on installed landing paths.

Replace the per-agent bespoke install formats with the Agent Skills open
standard: one canonical .agents/skills/<skill>/SKILL.md per project, read
directly by universal agents (Codex, Cursor, Cline, Gemini CLI, Copilot,
…) and via a relative symlink by every other agent. The bundled registry
grows from 8 hand-wired targets to the full standard set (72 agent ids).

BREAKING CHANGE: agent install no longer writes per-agent bespoke
formats. Codex no longer receives a managed section in AGENTS.md and
Windsurf no longer receives a size-capped compact body — both now read
the canonical .agents/skills/. The install result schema gains a `mode`
(canonical | symlink) field and `path` reflects the new model. Skills
written in the old format report stale/modified under `agent status`
until refreshed (re-run `agent install`, add --force where needed). The
legacy aliases (claude, kiro, copilot) and the claude-code default are
preserved, so existing commands keep working.
@coderabbitai

coderabbitai Bot commented Jul 22, 2026

Copy link
Copy Markdown

Review Change Stack

Walkthrough

Changes

The PR adopts canonical Agent Skills files under .agents/skills, classifies targets as universal or symlinked, adds aliases and migration handling, updates setup defaults to claude-code, and revises detection, tests, and documentation.

Canonical skills and target registry

Layer / File(s) Summary
Canonical skills and target registry
src/lib/agent-targets.ts, src/lib/agent-targets.test.ts, skills/*
Defines skill metadata, target classifications, aliases, canonical paths, frontmatter parsing, rendering, markers, and legacy layout metadata.

Canonical installation and status flow

Layer / File(s) Summary
Canonical installation and status flow
src/commands/agent.ts, src/commands/agent.test.ts, test/e2e/agent-install.e2e.test.ts
Writes canonical skill files, creates symlink or copy-fallback landings, handles force/backups and migration, and reports install/list/status results.

Setup defaults and command integration

Layer / File(s) Summary
Setup defaults and command integration
src/commands/init.ts, src/commands/init.test.ts, test/e2e/setup.e2e.test.ts
Uses claude-code as the setup default and updates setup wiring and canonical-path expectations.

Skill detection and doctor wiring

Layer / File(s) Summary
Skill detection and doctor wiring
src/lib/skill-nudge.ts, src/lib/skill-nudge.test.ts, src/commands/doctor.ts, test/e2e/skill-nudge.e2e.test.ts
Detects verification skills through supported landing-path existence and passes existsSync through doctor checks.

Documentation and contribution policy

Layer / File(s) Summary
Documentation and contribution policy
README.md, DOCUMENTATION.md, CONTRIBUTING.md
Documents target IDs, installation modes, setup usage, status behavior, and supported-agent maintenance requirements.

Estimated code review effort: 4 (Complex) | ~60 minutes

Sequence Diagram(s)

sequenceDiagram
  participant User
  participant AgentInstall
  participant CanonicalSkills
  participant AgentLanding
  User->>AgentInstall: agent install --target <id>
  AgentInstall->>CanonicalSkills: write or validate canonical SKILL.md
  AgentInstall->>AgentLanding: create symlink or copy fallback
  AgentLanding-->>User: install action and path
Loading

Possibly related PRs

Suggested reviewers: ruili-testsprite, zeshi-du

🚥 Pre-merge checks | ✅ 5
✅ Passed checks (5 passed)
Check name Status Explanation
Linked Issues check ✅ Passed The changes satisfy #270: canonical skills, universal/symlinked targets, legacy aliases, status, force overwrite, and path safety are all addressed.
Out of Scope Changes check ✅ Passed The diff stays focused on agent install/setup, docs, tests, and related skill files; no unrelated changes stand out.
Docstring Coverage ✅ Passed Docstring coverage is 83.33% which is sufficient. The required threshold is 80.00%.
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The title clearly summarizes the main breaking change: aligning agent skill installation with the Agent Skills standard.
✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@github-actions

github-actions Bot commented Jul 22, 2026

Copy link
Copy Markdown

✅ This PR is linked to an issue assigned to @powxenv — thanks! The needs-issue label has been removed.

@github-actions github-actions Bot added the needs-issue PR not linked to an issue yet — please open one first and claim it (see CONTRIBUTING) label Jul 22, 2026
powxenv added 2 commits July 24, 2026 14:07
…ehavior

- Revise CONTRIBUTING.md to describe universal vs symlinked agent targets and requirements for adding new targets
- Update DOCUMENTATION.md to clarify agent install command usage and differentiate universal and symlinked agents
- Expand README.md with a detailed supported agents table, including canonical ids, aliases, and skills folder locations
- Update src/lib/agent-targets.ts with added agents, corrections to skillsDir and universal flags, and alias mappings
- Modify unit tests and e2e tests to reflect renamed and updated agent targets, replacing deprecated aliases like gemini-cli with antigravity-cli and windsurf with devin-desktop
- Ensure single source of truth for skills at `.agents/skills`, symlink strategy for non-universal agents, and consistency across docs and code
@github-actions github-actions Bot removed the needs-issue PR not linked to an issue yet — please open one first and claim it (see CONTRIBUTING) label Jul 24, 2026
@powxenv
powxenv marked this pull request as ready for review July 24, 2026 12:09

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 3

🧹 Nitpick comments (1)
src/lib/agent-targets.test.ts (1)

131-165: 🎯 Functional Correctness | 🔵 Trivial | ⚡ Quick win

Coverage doesn't include adversarial alias tokens.

Tests confirm resolveTarget rejects a plain unknown token (line 156-158), but don't cover prototype-chain keys (constructor, __proto__, toString) that expose the bug flagged in agent-targets.ts (resolveTarget, lines 168-172). Once that's fixed, add a case here.

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@src/lib/agent-targets.test.ts` around lines 131 - 165, Add adversarial-token
coverage to the resolveTarget tests, specifically asserting that constructor,
__proto__, and toString are rejected as unknown tokens. Extend the existing
“resolveTarget rejects an unknown token” test in the TARGET_ALIASES +
resolveTarget suite, preserving the current null result expectation.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@DOCUMENTATION.md`:
- Around line 111-118: Update the agent install examples in the documentation to
pass each agent name through the documented --target option instead of as a
positional argument. Preserve the existing install, list, and status examples
unchanged in purpose.

In `@src/commands/agent.ts`:
- Around line 714-728: Update runStatus around the TARGETS and DEFAULT_SKILLS
iteration to emit universal ok rows only for targets explicitly requested by the
user or targets with an installed actionable artifact. Preserve rows for
actionable states, while suppressing ok-only universal results for uninstalled,
unspecified targets.

In `@src/lib/agent-targets.ts`:
- Around line 168-172: Update resolveTarget’s alias lookup to verify raw is an
own key of TARGET_ALIASES before reading its value, returning null for inherited
keys such as constructor, __proto__, and toString. Preserve the existing TARGETS
lookup and canonical alias behavior, and add regression coverage in
agent-targets.test.ts for these tokens to ensure unknown-target validation is
reached instead of a crash.

---

Nitpick comments:
In `@src/lib/agent-targets.test.ts`:
- Around line 131-165: Add adversarial-token coverage to the resolveTarget
tests, specifically asserting that constructor, __proto__, and toString are
rejected as unknown tokens. Extend the existing “resolveTarget rejects an
unknown token” test in the TARGET_ALIASES + resolveTarget suite, preserving the
current null result expectation.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Pro Plus

Run ID: 50a2a944-c182-480c-bf90-a91e921b1d6f

📥 Commits

Reviewing files that changed from the base of the PR and between fe07bc9 and 68f01d5.

⛔ Files ignored due to path filters (2)
  • package-lock.json is excluded by !**/package-lock.json, !package-lock.json
  • test/__snapshots__/help.snapshot.test.ts.snap is excluded by !**/*.snap, !**/*.snap
📒 Files selected for processing (18)
  • CONTRIBUTING.md
  • DOCUMENTATION.md
  • README.md
  • skills/testsprite-onboard.skill.md
  • skills/testsprite-verify.codex.md
  • skills/testsprite-verify.skill.md
  • src/commands/agent.test.ts
  • src/commands/agent.ts
  • src/commands/doctor.ts
  • src/commands/init.test.ts
  • src/commands/init.ts
  • src/lib/agent-targets.test.ts
  • src/lib/agent-targets.ts
  • src/lib/skill-nudge.test.ts
  • src/lib/skill-nudge.ts
  • test/e2e/agent-install.e2e.test.ts
  • test/e2e/setup.e2e.test.ts
  • test/e2e/skill-nudge.e2e.test.ts
💤 Files with no reviewable changes (2)
  • skills/testsprite-verify.codex.md
  • src/commands/doctor.ts

Comment thread DOCUMENTATION.md
Comment thread src/commands/agent.ts
Comment thread src/lib/agent-targets.ts
powxenv added 5 commits July 25, 2026 00:01
- Detect legacy own-file skill artifacts and AGENTS.md managed sections during install
- Back up legacy files and directories as *.bak before migrating or removing them
- Convert legacy skill folders (e.g., claude) to symlink landings pointing to canonical skills
- Remove managed section from AGENTS.md and back it up during migration
- Implement dry-run mode that plans migration without file changes
- Refuse plain installs when legacy artifacts block new symlink target paths
- Surface a status advisory listing legacy targets and how to migrate them
- Introduce findManagedSectionBounds utility to locate legacy sentinel blocks
- Update install action aggregation to consider migrated steps as updated
- Add comprehensive tests covering migration scenarios, error cases, and idempotency
- Extend filesystem mocks with readdir support for migration operations
- Update CLI help text for --force to mention legacy artifact migration and backups
- Update resolveTarget to use hasOwnProperty to exclude inherited prototype keys
- Add test to verify resolveTarget returns null for inherited keys like constructor, __proto__, toString, hasOwnProperty
- Ensure inherited prototype-chain keys are recognized as unknown targets and rejected explicitly
- Removed multiline conditional and combined it into a single line
- Maintained logic to check TARGET_ALIASES property existence before return
- Improved code readability and compactness without changing behavior
- Change `testsprite agent install` commands to use `--target` flag for specifying agents
- Update DOCUMENTATION.md examples to reflect new command syntax
- Modify README.md to replace old command usage with the new format
- Clarify onboarding instructions for different agents with consistent command style
…ked agents

- Update DOCUMENTATION.md to explain that 'agent status' checks each installed skill against CLI version
- Clarify that universal agents share one canonical skill file serving all agents
- Explain symlinked agents appear only when their own landing exists
- Update CLI help text to reflect these details for 'agent status' command
- Adjust snapshot test output to match the revised command description and behavior
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Hackathon] Support Agent Skills standard for install/setup

1 participant